INTERNET  DOCUMENT  INFORMATION  FORM 


A .  Report  Title:  Corporate  Executive  Information  System 


B.  DATE  Report  Downloaded  From  the  Internet:  10/20/99 


C.  Report's  Point  of  Contact:  (Name,  Organization,  Address,  Office 
Symbol,  &  Ph  #):  OAIG-AUD  (ATTN:  AFTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-2884 


D.  Currently  Applicable  Classification  Level:  Unclassified 

E.  Distribution  Statement  A:  Approved  for  Public  Release 

F.  The  foregoing  information  was  compiled  and  provided  by: 

DTIC-OCA,  Initials: _ ^VM _ Preparation  Date  10/20/99 


The  foregoing  information  should  exactiy  correspond  to  the  Titie,  Report  Number,  and  the  Date  on 
the  accompanying  report  document.  If  there  are  mismatches,  or  other  questions,  contact  the 
above  OCA  Representative  for  resolution. 


1 


vac  QiriuTy  UKncTsi)  4 


Additional  Copies 

To  obtain  additional  copies  of  this  audit  report,  contact  the  Second^  Reports 
Distribution  Unit  of  the  Analysis,  Planning,  and  Technical  Support  Directorate  at 
(703)  604-8937  (DSN  664-8937)  or  FAX  (703)  604-8932. 

Su^estions  for  Future  Audits 

To  suggest  ideas  for  or  to  request  future  audits,  contact  the  Planning  and 
Coordmtion  Branch  of  the  Analysis,  Planning,  and  Technical  Support  Directorate 
at  (703)  604-8939  (DSN  664-8939)  or  FAX  (703)  604-8932.  Ideas  and  requests 
can  also  be  mailed  to: 

OAIG-AUD  (ATTN:  APTS  Audit  Suggestions) 

Lispector  General,  DqKirtment  of  Defmse 
400  Army  Navy  Drive  (Room  801) 

Arlington,  Virginia  22202-2884 

Defense  Hotline 

To  rq>ort  fraud,  waste,  or  abuse,  contact  the  Defense  Hotline  by  calling 
(800)  424-9(^8;  by  sending  an  electronic  message  to  Hotline@DODIG.OSD.MIL; 
or  by  writing  to  the  Defense  Hotline,  The  Pentagon,  Washington,  D.C.  20301-1900. 
The  identity  of  each  writ^  and  caller  is  fully  protected. 


Acronyms 


MAIS 

OASD(HA) 

ORD 

PMO 


Major  Automated  Information  System 

Office  of  the  Assistant  Secretary  of  Defense  (Health  Affidrs) 

Operational  Requirements  Document 

Program  Management  Office 


INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-2884 


June  6,  1997 

MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  (COMPTROLLER) 

ASSISTANT  SECRETARY  OF  DEFENSE  (COMMAND, 
CONTROL,  COMMUNICATIONS,  AND 
INTELUGENCE) 

ASSISTANT  SECRETARY  OF  DEFENSE  (HEALTH 
AFFAIRS) 

DIRECTOR,  OPERATIONAL  TEST  AND  EVALUATION 
DIRECTOR,  PROGRAM  ANALYSIS  AND  EVALUATION 

SUBJECT:  Audit  Rqx)]t  on  Coipoiate  Executive  Diformation  System 
(Report  No.  97-152) 


We  are  providing  this  rqwrt  for  your  information  and  use.  This  is  the  third  of 
three  rqwrts  on  our  au<Ht  project  titled,  "The  Dq)artment  of  Defense  Health  Care  Cost 
Accounting  Systems." 

We  provided  a  draft  of  this  rqwrt  on  April  14,  1997.  Because  the  report 
contains  no  recommendations,  written  comments  were  not  required,  and  none  were 
recdved. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  Michael  A.  Josq>h,  Audit  Program  Director,  or 
Mr.  Sanford  W.  Tomlin,  Audit  Project  Manager,  at  (757)  766-2703.  See  Appendix  B 
for  the  report  distribution.  The  audit  team  members  are  &ted  inside  the  back  cover. 

David  K.  Steensma 
Dqiuty  Assistant  Inspector  General 
for  Auditing 


Office  of  file  Inspector  General,  DoD 

Report  No.  97-152  June  6, 1997 

(Project  No.  6LF-0047.02) 

Corporate  Executive  Information  System 
Executive  Summary 


Introduction.  The  Coipoiate  Executive  Information  System  (the  System)  developmrat 
began  in  June  1995  when  the  Army  Suti^eon  Gen^  was  designated  as  the  Executive 
Agent  The  System  is  intended  to  provide  customs  throughout  the  Military  Health 
Services  System  validated  clinical,  finandal,  managed  care,  and  administrative  decision 
siq)port  and  executive  information.  C^tomers  include  health  care  providers  and 
command  staff  within  military  treatment  facilities,  DoD  lead  agrats,  major  medical 
commands,  surgeons  graeral,  and  the  Office  of  the  Assistant  Secretary  of  Defense 
(Health  Af^s).  The  System  will  be  the  major  source  of  data  for  health  care  and 
budgeting  decisions. 

Audit  Objectives.  The  overall  audit  objective  was  to  determine  whether  DoD  health 
care  cost  accounting  systems  provide  managers  with  adequate  and  reliable  information 
for  cost-effective  health  care  and  budgeting  decisions.  During  the  audit,  the  Deputy 
Assistant  Secretary  of  Defense  (Health  Budgets  and  Programs)  r^uested  ftat  we  delay 
our  review  of  the  cost  accounting  systems  because  seve^  initiatives  were  underway  to 
improve  DoD  health  care  automata  information  systems.  Therefore,  we  limited  our 
au^t  coverage  to  the  devel^ment  of  the  System.  We  also  evaluated  the  managemrat 
control  program  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  as  it  ^lied  to 
the  System. 

Audit  Results.  The  System  was  not  classified  as  a  major  automated  information 
system,  and  its  life-cycle  cost  was  not  adequately  estimated  and  reported.  As  a  result, 
developmrat  risks,  such  as  not  meeting  the  needs  of  System  usmrs;  slipping  dq)loymrat 
schedides;  incurring  additional  cost  due  to  delays  in  the  shutdown  of  existing  systems; 
and  the  System  not  represrating  the  best  value  solution  for  meeting  user  requiremrats, 
were  not  mitigated.  The  Program  Managemoit  Office  initiated  action  to  reduce  the 
risks  discussed  in  this  r^rt.  The  most  significant  action  was  the  transfer  of  System 
approval  authority  to  the  Major  Automated  Information  System  Review  Council  on 
(jctober  17,  1996.  In  our  opinion,  classifying  the  System  as  a  major  automated 
information  system  and  the  resulting  increaW  focus  on  program  management  has 
established  tiie  key  controls  necessary  to  reduce  risks  associated  with  system 
develqpm^t.  Therefore,  we  are  not  making  recommradations  in  this  report.  See 
Part  I  for  details  of  the  audit  results  and  Appradix  A  for  details  on  the  management 
control  program. 

Management  Comments.  We  provided  management  a  draft  of  this  rq>ort  on 
April  14,  1997.  Because  the  r^rt  contains  no  recommendations,  written  comments 
were  not  required,  and  none  were  received. 
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Part  1  -  Audit  Results 


Audit  Results 


Audit  Background 


The  Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  (OASD[HA]) 
is  re^nsible  for  the  effective  execution  of  the  DoD  health  care  mission.  The 
mission  includes  providing  top  quali^  health  services  whenever  needed; 
supporting  mili^  operations;  and  providing  s^ces  to  members  of  the  Armed 
Forces  and  thdr  dq)endrats  and  to  odi^  entitled  to  DoD  health  care.  To 
efficiently  carry  out  this  mission,  the  OASD(HA)  has  adopted  a  ^stematic 
apiHoach  to  eliminate  unnecessaiy  duplication  of  health  care  information 
systems.  This  {q)proach  includes  migrating  die  necess^  functionality  of 
multiple  legacy  (^sting)  systems  into  one  target  executive  information  and 
decision  support  system.  The  Corporate  Executive  Information  System  (the 
System)  is  the  target  system  that  will  support  medical  treatment  facilities,  dental 
treatment  £a(^ties,  lead  agents,  the  Military  Departments,  and  other  DoD 
users. 

System  developm^t  began  in  June  1995  when  the  Principal  Dqiuty  Assistant 
Secretary  of  Defense  (Health  Affairs)  si^ed  a  contract  designating  the  Army 
Surgeon  Graeral  as  the  System's  Executive  Agoit.  According  to  the  contract, 
the  Military  Health  Services  System  Proponent  Committee  serves  as  the 
milestone  dedsion  authority  and  approves  System  functional  requirem^ts.  The 
System  is  intenddi  to  be  an  executive  information  and  decision  support  package 
that  presents  data  collected  from  a  variety  of  information  systems.  It  will  be 
designed,  developed,  dqiloyed,  and  implemented  in  two  major  phases,  near 
term  and  ^  term.  Near-term  products,  initially  scheduled  for  deployment 
beginning  in  July  1996,  will  focus  on  absoption  of  functionality  from  eight 
existing  systems,  with  limited  new  functionality.  Far-term  products,  scheduled 
for  deployment  b^inning  in  October  1998,  provide  DoD  managers  witii 
medical  information  needed  to  make  health  care  and  budgeting  decisions.  For 
example,  the  System  will  provide  data  to  assist  in  decisions  relating  to  capitation 
budgeting,  contract  bidprice  adjustments,  utilization  management,  and  other 
areas. 


Audit  Objective 


The  overall  audit  objective  was  to  determine  whether  DoD  health  care  cost 
accounting  systems  provided  managers  with  adequate  and  reliable  information 
for  cost-effective  health  care  and  budgeting  decisions.  During  the  audit,  the 
Deputy  Assistant  Secretary  of  Defense  (Health  Budgets  and  Programs)  requested 
that  we  delay  our  review  of  cost  accounting  systems  because  several  initiatives 
w^e  underway  to  improve  DoD  health  care  automated  information  systems. 
Therefore,  we  limited  our  audit  coverage  to  the  development  of  the  System. 
We  also  evaluated  the  management  control  program  of  the  Assistant 
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Audit  Results 


Seoretaiy  of  Defense  (Health  Affairs)  as  it  applied  to  the  System.  See 
Appenduc  A  for  a  discussion  of  the  scope  and  methodology  and  for  details  of 
our  review  of  the  managem^t  control  program. 
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The  Corporate  Executive  Information  System  (the  System)  was  not 
classified  as  a  major  automated  information  system  (M^S),  and 
life-cycle  cost  was  not  adequately  estimated  and  reported.  This  occurred 
because  controls  w^  not  in  place  to  define  and  manage  user 
requiremrats.  As  a  result,  significant  devdopment  risks,  such  as  not 
meeting  the  needs  of  System  users;  slicing  dq)loyment  schedules; 
incurring  additional  cost  due  to  delays  in  the  shutdown  of  existing 
systems;  and  the  System  not  rqyresoiting  the  best  value  solution  for 
meeting  user  requirements,  were  not  mitigated.  Recent  management 
action  has  established  key  controls  necessary  to  reduce  risks  associated 
with  system  develcpment. 


Criteria 


DoD  Directive  5000.1.  DoD  Directive  5000. "Defense  Acquisitions,” 
htoch  15,  1996,  provides  broad  policies  and  prindples  for  aU  DoD  acquisition 
programs,  and  establishes  a  disciplined,  yet  flexible,  management  approach  for 
acquiring  qudity  products.  The  Directive  establishes  responsibilities  for  DoD 
Componrat  heads  that  includes  ensuring  that  polides  and  procedures  governing 
the  operation  of  the  Component's  acquisition,  budgeting,  and  requirements 
systems  are  effectivdy  implemented.  The  Directive  summarizes  the  primary 
objective  of  a  defense  acquisition  as: 

...  to  acquire  quality  products  that  satisfy  the  needs  of  the 
operational  user  >vith  measurable  inqirovements  to  mission 
acconqilishment,  in  a  timely  manner,  at  a  &ir  and  reasonable  price. 

Successful  acquisition  programs  are  fundamentally  d^ndent  upon 
conq>etent  people,  rational  priorities,  and  clearly  defined 
responsibilities,  llie  following  policies  and  principles  govern  die 
operation  of  the  defense  acquisition  system  and  are  divided  into  three 
major  categories:  (1)  Translating  Operational  Needs  into  Stable, 

Affordable  Programs,  (2)  Acquiring  Quality  Products,  and 
(3)  Organizing  for  Efficiency  and  Effectiveness.  These  principles 
shall  guide  all  defense  acquisition  programs. 

DoD  R^plation  5000.2-R.  DoD  Regulation  5000.2-R,  "Mandatoiy 
Procedures  for  Major  Defense  Acquisition  Programs  (hTOAPS)  and  Major 
Automated  Information  System  (MAIS)  Acquisition  Programs," 


^Effective  March  15,  1996,  DoD  Directive  5000.1  and  DoD  Regulation 
5000.2-R  consolidate  acquisition  guidance  previously  provided  under  DoD 
Directives  5000.1  and  8120.1.  The  overall  concepts  and  requirements 
established  in  prior  guidance  are  consistent  with  requirements  in  the  new  DoD 
directive  and  regulation. 
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htoch  IS,  1996,  establishes  mandatory  procedures  for  MAIS  acquisition 
programs.  It  defines  a  MAIS  acquisition  program  as  an  automated  information 
system  promm  tiiat  is  estimated  to  require  program  costs  in  any  single  year  in 
excess  of  $30  million;  total  program  costs  in  »cess  of  $120  million;  or  total 
lifeeycle  cost  in  excess  of  $360  million.  The  regulation  rq>lac^  earlier 
guidance  that  contained  lower  cost  thresholds.  It  requires  ma^ement  to 
structure  the  MAIS  to  oisure  a  logical  progression  through  a  series  of  phases 
designed  to  reduce  risk,  ensure  affordability,  and  provide  adequate  information 
for  dsdsionmaldng  that  will  provide  tiie  nera  in  the  shortest  practical  time. 


Designation  as  a  Mm*or  Automated  Information  System 


The  System  was  not  classified  as  a  MAIS  until  October  1996.  As  total  system 
requirements  grew,  the  cost  thresholds  that  identify  a  MAIS  candidate  were 
surpassed.  From  October  1995  to  Sq)tembCT  1996,  changing  requiremaits 
resulted  in  total  life-cycle  cost  estimates  varying  from  $110  million  to 
$362  million.  In  June  1996,  the  OASD(I^)  had  obligated  $54  million  in 
FY  1996  System  developmoit  funds,  exce^ing  the  $30  inMon  p»  year  MAIS 
requirement.  We  could  not  det^mine,  from  available  documentation,  at  what 
point  before  the  actual  expenditure  of  funds  tiie  Program  Managemrat  Office 
(PMO)  became  aware  that  the  MAIS  cost  thresholds  would  be  surpassed. 


Estimating  and  Reporting  Life-Cycle  Cost 


The  PMO  did  not  adequately  estimate  and  rqwrt  the  System's  life-cycle  cost. 
Life-cycle  cost  is  the  total  cost  to  the  Government  for  me  System  ov&t  its  full 
life.  It  includes  the  cost  of  requiremrats  analysis;  design;  develr^moit; 
acquisition  and  lease;  operations;  support;  and  wh»e  applicable,  disposal.  The 
life-cycle  cost  is  odti^  to  many  tiicets  of  program  managem«it,  such  as 
evaluating  tiie  System's  cost-effectiveness  and  determining  the  amount  and 
timing  of  timding  requirements.  Estimating  and  rqiorting  accurate  cost  are 
necessary  for  successful  System  control  and  development. 

Estimated  Cost.  Life-cycle  cost  of  the  System  was  not  adequately  estimated. 
The  PMO  did  not  have  the  capacity  and  usage  information  necessary  to 
accurately  determine  System  cost.  For  example,  the  cost  to  integrate  the 
System  within  the  existing  Militaty  Health  Services  System  was  not  determined 
because  the  number  and  capabilities  of  the  computers  at  the  corporate  usors, 
lead  agents,  and  military  treatment  facilities  were  unknown.  In  addition,  the 
capacity  and  usage  of  the  existing  communications  infrastructure  of  the  Military 
H^th  Services  System  was  unknown.  Estimating  the  cost  to  integrate  the 
System  requires  comparing  hardware  and  software  requiremrats  with  the 
existing  computer  capability.  Such  comparison  could  not  be  done  because  the 
System  user  requirements  had  not  been  sufficiently  defined. 
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Reported  Cost.  The  OASD^IA)  did  not  adequately  report  System  life-cycle 
cost  as  rrauiied  by  DoD  policies  and  procedures.  DoD  Directive  5000. 1  states 
that  acquisition  systems  should  translate  operational  needs  into  stable,  affordable 
prognms.  Also,  information  technology  resources  required  to  support  an 
acquisition  progrm  should  be  included  in  the  budget  submission  exhibits. 
OASD(HA)  budget  estimate  submissions  varied  agnificantiy  from  OASD(HA) 
estimates  of  System  life-cycle  cost.  The  following  table  shows  tiie  fluctuation  in 
budget  requests  (budget  estimate  submission)  and  estimated  System  life-cycle 
costs  (system  hct  sheet  and  functional  economic  analysis). 


Reported  Life-Cycle  Costs 


Document 

Dalg 

Estimated 
Life-Cycle  Cost 
fin  n^onsl 

Budget  estimate  submission 

October  1995 

$109.8 

System  fact  sheet 

June  1996 

326.7 

System  fact  sheet 

September  1996 

301.3 

Functional  economic  analysis 

Sq)tember  1996 

362.5 

Budget  estimate  submission 

October  1996 

192.4 

System  fact  sheet 

February  1997 

374.9 

The  System  fact  sheet  dated  June  1996  stated  that  additional  requirements  will 
be  fully  funded  by  the  OASD(HA)  and  the  Services.  However,  other  health 
programs  could  be  put  at  risk  by  having  to  absorb  a  System  funding  shortfall. 
For  example,  the  budget  estimate  submission  dated  October  1996  is  about 
$180  mMon  less  than  the  life-cycle  cost  included  in  the  System  fact  sheet  dated 
February  1997.  To  ensure  the  affordability  of  the  System,  reported  cost 
estimates  need  to  incorporate  all  anticipated  requirem^its. 


Defining  and  Managing  User  Requirements 


The  System  was  not  designated  as  a  MATS,  and  life-cycle  cost  was  not 
adequately  estimated  and  reported  because  controls  were  not  in  place  to  define 
and  manage  us^  r^uirements.  User  requirements  are  one  of  the  major  cost 
fcictors  in  the  acquisition  of  an  automated  information  system.  Requirements 
impact  software  and  hardware  developmrat,  deployment  schedules,  user 
training  documentation,  and  the  Support  of  the  system  ^oughout  its  life  cycle. 
It  is  essratial  to  toe  successful  acquisition  of  an  automated  information  system 
that  requiremrats  are  sufficiently  defined  and  managed. 

Defining  Requirements.  Controls  over  defining  user  requiremrats,  which 
determine  the  minimum  operational  capability  of  toe  system,  w^  not  adequate. 
DoD  Directive  5(X)0.1  requires  that  at  each  milestone,  beginning  with  program 
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initiation  (usually  milestone  I),  thresholds^  and  objectives^  be  defined  for  cost, 
schedule,  and  performance.  DoD  R^ulation  5000.2-R  describes  how  the  data 
can  be  i»esented  in  an  operational  requirements  documrat  (ORD).  As  of 
Janua^  1997,  the  PMO  did  not  have  an  approved  ORD  and  the  System  was  in 
the  milestone  n  (ragineeting  and  develt^ment)  phase  of  the  acquisition  process. 
The  draft  ORD,  dated  July  23,  1996,  ^d  not  adequacy  define  user 
requirements.  For  example,  the  draft  ORD  did  not  include  the  number  of 
operating  units  (computers  and  servers)  needed  for  System  dq>loyment.  Also, 
the  schedule  considerations  included  in  the  draft  ORD  ^d  not  clearly  specify  the 
operational  capability  or  level  of  performance  needed  for  initial  and  folly 
<^)etational  ca^ility. 

Managing  Requir^ents.  The  PMO  did  not  have  the  controls  in  place  to 
effectively  manage  new  requirements  and  assess  their  impact  on  System  cost, 
schedules,  and  capabilities.  Monitoring  baseline  requirements  fen:  each  function 
that  satisfies  user  and  int^ce  requirements  is  a  technique  for  controlling  die 
development  of  a  system  through  a  formal  management  process.  Baseline 
requirements  should  also  identify  die  completion  of  major  milestone  activities. 
In  addition  to  not  having  adequately  defined  requiremoits,  the  PMO  accepted 
additional  requirements  without  assessing  their  impact  on  the  System's 
developm^t  as  shown  bdow. 

o  In  the  first  quarts  of  FY  1996,  the  hfilitary  Health  Services  System 
Proponent  Committee  required  that  the  System  be  available  in  all  military 
treatment  focilities  at  the  clinical  level  instead  of  at  the  command  level  as 
originally  planned.  This  change  in  requirements  increased  the  estimated  number 
of  System  users  from  3,000  to  7,S(X).  Documoitation  was  not  available  to  show 
how  this  requirement  affected  System  cost,  deployment  schedules,  and 
capabilities. 

o  The  PMO  stated  that  the  Military  Health  Services  System  Proponent 
Committee  further  directed  that  the  System  would  be  used  for  the  Medicare 
subvention  demonstration  project.  Diat  lequiremrat  should  have  been 
incorporated  into  the  ORD.  Further,  the  additional  capability  significantly 
increased  the  technical  and  financial  risks  to  the  System  development  because 
the  scope  of  the  System  was  increased.  As  of  October  1996,  requirements  for 
the  Medicare  subvention  demonstration  project  were  t^g  determined. 
Howev^,  the  PMO  anticipated  that  tiie  System  would  be  required  to  present 
patient  level  cost  allocation  data,  which  is  an  entirely  new  fonctionaHty  to  the 
Military  Health  Services  System.  The  effect  of  a  new  functionality  on  System 
cost  and  development  schedule  needed  to  be  evaluated  before  incorporating  the 
requiremoits  into  the  System. 


^The  threshold  is  the  minimum  acceptable  value  that,  in  the  user's  judgment,  is 
necessary  to  satisfy  the  need.  If  threshold  values  are  not  achieved,  program 
performance  is  seriously  degraded,  the  program  may  be  too  costly,  or  may  be 
untimely. 

^The  objective  is  a  value  that  is  desired  by  the  user  and  one  that  the  program 
manager  is  attempting  to  obtain. 
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The  PMO  did  not  establish  procedures  to  assess  the  impact  and  to  control  the 
integration  of  new  requirements.  One  mediod  to  systematiadly  monitor  changes 
is  through  a  configuration  control  board.  The  board  controls  changes  by 
reviewing  and  approving  modifications  to  the  baseline  configuration.  Although 
a  configuration  control  board  was  discussed  in  the  program  management  plsm, 
the  program  manager  was  not  involved  with  the  board  and  we  did  not  find  any 
evidence  of  a  functioning  board.  As  of  October  1996,  the  System  contractor 
was  not  actively  partidpating  in  configuration  management  because  the 
contractor  no  longer  had  the  staff  and  software  it  previously  had  devoted  to  the 
function.  In  October  1996,  the  PMO  and  the  contractor  started  formalizing 
procedures  to  implement  a  working  configuration  control  board.  Establishing  a 
configuration  control  board  would  have  established  a  frame  of  reference  to 
monitor  requirement  changes  and  document  their  effect  on  System  development. 


Mitigation  of  System  Risk 


Without  the  controls  in  place  to  effectively  manage  the  initial  and  additional 
requirements,  the  fiscal  and  technical  risks  of  developing  a  system  that  meets 
users'  needs  are  greatly  increased.  Completion  of  near-term  deployment  has 
slipped  from  the  end  of  FY  1997  to  the  third  quarter  of  FY  1998  due  to  chwges 
in  requirements.  The  slippage  delays  the  System's  absorption  of  the  functions 
of  existing  systems,  which  have  an  annual  opmting  cost  of  $14  million.  Li 
addition,  budgeting  for  less  than  the  total  estimated  cost  is  not  consistent  with 
DoD  policy;  and  it  places  System  acquisition  at  risk.  Assessing  the  effect  of 
orig^  and  additional  requiremoits  on  System  life-cycle  cost  is  necessary  to 
continually  evaluate  the  System's  affordability  and  to  oisure  Aat  it  rqiresrats 
the  best  value  solution. 


Management  Initiatives  to  Reduce  System  Risks 


The  PMO  has  taten  action  to  reduce  the  risks  discussed  above.  Prindpal  staff 
assistants  of  MAIS  Review  Council  members  from  the  Offices  of  the  Under 
SeCTetary  of  Defense  (Comiitroller)  and  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications,  and  Intelligence)  and  the  Directors  of 
Operational  Test  and  Evaluation  and  Program  Analysis  and  Evaluation 
concurred  with  our  concerns  about  unmitigated  System  development  risks.  The 
most  significant  result  from  actions  the  PMO  took  was  the  transfm:  of  System 
approved  authority  to  the  MAIS  Review  Council  on  October  17,  1^6. 

In  conjunction  with  die  System's  MAIS  designation,  the  PMO  has  takoi  steps  to 
comply  with  project  documentation  requiremoits  outlined  in  DoD  Directive 
5000.1  and  DoD  Regulation  S0(X).2R.  The  PMO  is  quantifying  operational 
requirements  and  their  impact  on  system  development  and  cost.  ’!^e  PMO  has 
developed  a  revised  draft  ORD  and  distributed  it  to  die  MAIS  Review  Coundl 
for  comment.  Also,  the  PMO  is  working  with  the  Office  of  the  Assistant 
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Secretary  of  Deftaise  (Command,  Control,  Communications,  and  Intelligence) 
to  develop  an  acquisition  planning  baseline  that  includes  details  on  project  cost, 
schedule,  and  pOTormance.  A&r  tiie  baseline  is  approved,  Ae  PMO  will 
establish  System  milestone  points  and  timelines.  Since  becoming  a  MAIS,  the 
PMO  has  develo]^  a  draft  test  and  evaluation  master  plan  that  the  Office  of  the 
Director,  Opraational  Test  and  Evaluation  is  reviewing.  The  plan  includes 
provisions  for  using  a  contractor  who  is  indq)radent  of  the  System  contractor  to 
conduct  devdopmental  testing.  The  PMO  is  also  developing  complete  and 
accurate  life-cycle  cost  estimates.  Under  the  advisemoit  of  the  Office  of  the 
Director,  Program  Analysis  and  Evaluation,  the  PMO  is  contracting  with 
industry  exp^  to  prepare  an  indq)endent  component  cost  analysis,  hi  our 
opinion,  classifying  the  System  as  a  MAIS  and  the  resulting  increased  focus  on 
program  management  has  established  the  key  controls  necessary  to  ensure  that 
us^  requirements  are  suffidentiy  defined  and  managed.  In  addition,  risks 
associate  with  system  development  are  bdng  reduced.  As  a  result,  we  are  not 
making  any  recommendations  in  this  rqxirt. 
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Appendix  A.  Audit  Process 


Scope  and  Methodology 


We  reviewed  DoD  policies  on  information  system  acquisition.  We  also 
reviewed  procedures  at  the  OASD(HA)  and  the  PMO,  and  System 
documentation  from  June  1995  to  October  1996.  We  also  reviewed  the  budget 
execution  summaries  for  FYs  1997  through  1999,  submitted  by  OASD(HA)  to 
the  Office  of  the  Under  Secretary  of  Defense  (Comptroller). 

Use  of  Computer-Processed  Data.  We  did  not  rdy  on  computer-processed 
data  to  p^orm  this  audit. 

Limitations  to  Audit  Scope.  The  overall  audit  objective  was  to  determine 
whether  DoD  health  care  cost  accounting  systems  provide  managers  with 
adequate  and  reticle  information  for  cost-effective  he^th  care  and  budgeting 
decisions.  The  OASD(HA)  recognized  tiiat  its  automated  systems  did  not 
provide  managers  with  adequate  information  and  initiated  developmoit  of 
seva^  automated  systems.  In  a  memorandum  dated  May  8,  1996,  the  Deputy 
Assistant  Secretary  of  Defense  (Health  Budgets  and  Programs)  recommended 
that  we  delay  Ae  audt  because  of  sev»:al  iiutiatives  underway  to  improve  DoD 
health  care  automated  information  systems. 

Although  the  System  is  not  an  accounting  system,  it  is  intraded  to  be  an  integral 
component  of  the  health  care  and  budgeting  decision  process  by  providing 
managers  with  financial  and  clinical  information.  Therefore,  we  focused  our 
audit  on  evaluating  tiie  managemoit  of  the  development  of  the  System. 

Use  of  Technical  Assistance.  Our  Readiness  and  Operational  Support 
Directorate  and  Technical  Assessmoits  Division  assisted  us  in  evaluating 
technical  documentation. 

Audit  Periods  and  Standards.  We  performed  this  jprogram  audit  from  March 
1996  through  February  1997  in  accordance  with  auditing  standards  issued  by  the 
Comptroll^  General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD.  The  audit  included  such  tests  of  management  controls 
considered  necessary. 
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Contacts  Dur^  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  widim  DoD.  Furth^  details  are  available  on  request. 

Sununaiy  of  Prior  Audits  and  Other  Reviews.  During  the  last  S  years,  there 
were  no  prior  audits  or  reviews  of  the  System. 


Management  Control  Program 


DoD  Directive  5010.38,  "hitemal  Mai^emoit  Control  Program,"  April  14, 
1987*,  requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  the  Review  of  ttie  Management  Control  Program.  We  evaluated 
management  controls  related  to  the  System  developmoit.  Specifically,  we 
evaluated  the  OASD(^)  implemmtation  of  DoD  policies  and  procedures 
governing  die  acquisition  of  MAIS.  We  reviewed  the  results  of  any 
self-evaluation  of  those  management  controls. 

Adequacy  of  Management  Controls.  At  the  OASD(HA),  we  identified 
material  management  control  weaknesses  in  System  development  as  defined  by 
DoD  Directive  5010.38.  Management  controls  within  the  OASD(HA)  did  not 
ensure  that  die  System  was  classified  as  a  MAIS  and  that  life-cycle  cost  was 
adequately  estimated  and  rqiorted.  The  details  of  the  management  control 
weaimesses  are  discussed  in  Part  I  of  this  rqiort.  However,  this  report  does  not 
contain  recommradations  because  the  PMO  took  the  necessary  actions  to 
re^nd  to  our  concerns.  A  copy  of  the  final  report  will  be  provided  to  the 
senior  official  in  charge  of  managemrat  controls  for  OASD(HA). 

Adequacy  of  Managonent's  Self-Evaluation.  Although  the  OASD(HA) 
established  an  Information  Managemoit  Project  Review  Board  as  one  initiative 
to  improve  management  controls  over  mission-related  activities,  it  did  not 
initiate  actions  to  correct  identified  weaknesses.  One  of  the  board's  functions  is 
to  p^orm  program  reviews  using  established  functional,  programmatic,  and 
technical  criteria.  In  June  1996,  the  board  conducted  an  evaluation  of  the 
System  and  found  teat  tee  System  should  be  considered  for  designation  as  a 
MAIS.  The  evaluation  also  identified  a  possible  increased  technical  risk  due  to 
tee  imposition  of  new  requirements. 


*DoD  Directive  5010.38  has  been  revised  as  "Management  Control  (MC) 
Program,"  August  26,  1996.  The  audit  was  performed  under  tee  April  1987 
version  of  tee  Directive. 


13 


Appendix  B.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secrets^  of  Defense  (Comptroller) 

Dq)uty  Chief  Financial  Office 
Dq)uty  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defuse  (Command,  Control,  Communications  and  Intelligence) 
Assistant  Seraetary  of  Defuse  (Health  AfGurs) 

Assistant  Secretary  of  Defense  (^blic  Af^s) 

Director,  Defense  Logistics  Studies  Information  Exchange 
Director,  Operational  Test  and  Evaluation 
Director,  Program  Analysis  and  Evaluation 


Department  of  the  Army 

Auditor  Gmieral,  Dqpartment  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Auditor  General,  Dq>artm©nt  of  the  Navy 
Superintendent,  Naval  Postgraduate  School 


Department  of  the  Air  Force 

Assistant  Seo^tary  of  the  Air  Force  financial  Management  and  Comptrollm*) 
Auditor  General,  Dq)artment  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agracy 
Director,  National  Security  Agracy 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
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Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Managemrat  and  Budget 
General  Accounting  Office 

National  Security  and  Interoational  Affidrs  Division 
Technical  Information  Goiter 
Health,  Education,  and  Human  Services 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Soiate  Committee  on  Apprqiriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Soiate  Committee  on  Govemmoital  Affiirs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Govemmoit  Reform  and  Oversight 
House  Subcommittee  on  Government  Management,  Information,  and  Technology, 
Committee  on  Govemmrat  Reform  and  Ov^ght 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Govemmoit  Reform  and  Oversight 
House  Committee  on  National  Security 
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